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ABSTRACT’ / 
A study involving six high risk chilireyg (K through 
grajie 2) was conducted to determine the effectiveness of a ‘ 
‘methodology for validating identification and placement of students 
diagnosed as educable mentally retarded, specific learning disabled, 
and emotionally disturbed. The methodology wsed was es 
Criterion-referenced tests in a time-series design; and evidence of 
th2 effectiveness of identification and placement was based on two 
patterns in the data--an increase in the score gains after placement 
and an abrupt change in the achievement score at the time of 
.placement as well as change in learning rate. Of the six children, 
five exhibitea@Tearning rates after placement that were higher than 
learning rates prior to placelent. (Advantages and disadvantages of 
+he methodology are briefly discussed.) (SBH) 
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A Methodology’ ee Selidating Piavanent of children 
, ee in aie maine Chiid: Pebgname : 


Placement of children identified as Edu¢able Mentally Retarded 


or Learning ‘Disabled into apeeint ty. equipped and staffed educational 


settings such as self- contained or Yesource rooms requires additional 4 
financial resourees that may be justifiably questioned by looal«fundina 
agEnCIEE A sound research nas mus t be. established to justify she” 


sadtvicnal resources necessary for transfer of exceptional children. re 


into special education settings. ae 


Research in the field.is frequently restricted by! the inability 


of researchers to assign children randomly to experimental and control 
groups and the legislative mandates requiring, agencies to provide 
services to all children in need. Other. constraints are the 5 


’ 7 f a H 
inadeauacies of non-equivalent control group research designs for 


‘populations in which regression and selection are strong threats to 


internal validity and the lack of sensitivity of commonly administered 


norm-referenced measures for detecting academic gains in the exceptional 
; | qe 
child popwulatigns. J 


The evaluation of the Developing Models for Special. Fducation, ts 


atts III/1IV-C Project encountered. all of ‘the previously mentioned 


limitations in Ydentifying a methodologically-sound evaluation design 


° for determining’ the effectiveness of the program: The DMSE Project was ———~ 
1 A 


4 - eee 
funded to develop a special education delivery system for the early 


identification of and intervention with children exhibiting three 

or one exceptionalities, Educable Mentally Retarded. Specific 
Learning Disabled and Emotionally Disturbed, , The bavget population oe 
consisted of K-2 childrén in an educationally and culturally¥ 


je 


disadvantaged environment. 9 ° 
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/ ‘ ok Methodology ee ace 
ae an 2g Sieg i 
. The me eHOnOLIay deterniived most feasible doz the evaluation of 
the DMSE Pragect, utilized criterion- tegurdhced’ tests ina time-series 
onion: The basic approach involyed the establishment of individual 
behavioral baselines defined by pene munca abi aetives with experimental 
7 replications across individuals: Use of this methodology Statutes the 
‘effects of identification and intervention on the children entering the 
exceptional child services ‘system during the 1975-1976 academic year. 
rior to. the beg inndng Sf atlico?, thirty-seven high-risk candidates 
for the program were identified by project personnel on the basis of 
; "pre-school ° screening, asbup aentey pei and group IQ data. - Three alternate 
forms of a criterion-referenced xést were developed to measure randomly 
+ selected academic behavioral sbivotives identified by teachers to be 
taught forthe warded Rouiiehion and to regular classroom K-2 children 
aes the. course of. the 1975-1976 academic year. These tests were 
A aaiiekeind monthly by project ‘personnel on an individual euatacts the 
37 high-risk danaiasees, Learner performance on the alternate forms 
_ was recorded priodr to and subsequent to identification and placement. — 
If the tdentigiextion and intervention process sia effective, the 
perfdrmance of handicapped echildren subsequent to placement would be 
superior to performance prior to placement. 
The evaluation design for the project was as follows: ‘ 


| 


with X; (placement) occurring anywherebetween 02 and 07 depending upon 


progress of the individuals through the ‘system. Thus, tegting occasions 


were pre-set and equally spaced but the application of the treatment was 


Cd 


oe ee ae ‘ ; , F aes. 
case-Specific instead of predetermined as in a typical time-series design. 
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' nig Two patterns in ‘the data were to be accepted as evidence of the. 
| f ' ite 
effectiveness of identificMtion: and placement. First, a’ change in the 
> 9 
raté of learning progress as demonstrated by an TOGEESEE in the score 


gains after placénent would indicate a beneficial pladshent for the . 


child. Figure 1 presents a hypothetical learning pattern demonstrating 
“Seething rate change. Second, an abrupt change in the ‘achievement 

score at the time of placement as well as a‘change in learning rate 

would indicate an advantage for. the child in placement. Figure 2 presents 

a hypothetical learning pattern demonstrating ,an abrupt change at the 

s . _ time of placement in conjunction with a learning cate change: Obviously, 

the most ideal learning vattern would be one that exhibited both a shift 


a - f F ; , 
in rate and an abrupt change upon placement. . _ 
, ( ; a. 
i Test Development 
/ 
The three alternate forms of a criterion- permeaieed: test it based 


on objectives. to be taught in indanenttan: @eeee grade, hastens grade, 
> 


and@xceptional child programs. Objectives drawn from the county list 
of ‘objectives and-from the Santa Cruz Behavioral Characteristics 
Progression Chart were reviewed by the teachers. ‘Approximately ‘25% of 


«+ the objectives dn which consensus was obtained were randomly selected 
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to be included on tha tests. Each! tonsensual objective was directly - * 
. - : ' 7 ‘ . 


measured on eactt of the three ‘alternate forms of the test. Most of the 5 
‘c objectives for the target population. were very fundamental with fairly = 
S restricted parameters for item construction. Objectives and coprespond- 
“ing eens ware ae to two external judges to. provide a check on 
. the validity of the item® used in ee evaluationy: ie 
The method of determining the pet yaneleey of the items tiked. in the : 
evaluation was multiple observations of “we testing of individual pupils 
s and consisted of agreement between observers. " Gonsistency of agreement 
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was examined. during the field testing of ‘the -instrument« Ninety-six 


percent of the multiple observations (133/138) ‘resulted in 100% 


agreement amang the four observers. hc8 


“An index of internal consistency was computed for the first form 


BE chk test (KR-20) which had a value ‘of .99.: This value is “extremely 


eG 8 pir } 
high for ie index, which ranges, from, .00 to 1.00. The index, however, . 


‘is sensitive tq increases in she number of items includéd on the test *% 


and to wide differences in the testing: population. With 118 items on 


the test and a range of EMR-five year-olds to ED seven-year-olds,- one 


\ . 


would aeoect a high degree of disery mination. 


Correlations were cAlculated among “the first four (administrations . 


shine the test in: ardee to estimates the reliability of the three a 


Foune that were developed. These correlations are underestimates of-the 


-, true reliability that exists amaqng the measures. since in each case a 


,988, .989, and .984. 


month elapsed between administrations during which time changes were 


anticipated in the. performance of students. -The correlations were: oe 
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Results and Discussion 


- Of the’ 37 high-risk children, ‘six were “actually placed ‘in, 
exceptional :child settings during the school year. Two methods, of 
NEARY the dataewere presented. First, examination of the raw seuyeu } 
pigeted anes testing time visually demonstrated oe effect of 


identification and placement for each ehiia. second , regression- 


.discontinuity was ‘employed as an tins to visual analysis. ror 


‘ 3 { ) 
each child best-fitting lifes were calculated for the pre-. and post~- 


i; 
placement scores. Comparisons were made of the Blopes and \intercepts . 


. ef Ene: “Lines paige to and following Placement. 
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Table 1 DECREUAE the individual scores on the achievement, measure. 
Figures 3-8 present by case numbers, the individual achievement prt ales 
developed to demonstrate visually the learning performance exhibi ed 


during the 1975-1976 academic year. - 
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ae: i 7 Table 1 


Individual Scars for Children Placed during 1975- 1976 


i Testing ; 
Period Case Number 


2 NT, indicates not tested 
ae 4 
--- ‘indicates time,of placement’! 
a 
Table 2 presents for each case the intervention intereepts and the 
slopes of regression lines cakculated' on date points prior to placement 


and following placement. 
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‘ : Table 2. ~ : 
Intervention Intercepts and Slopes of Regression \Lines 
Prior To And»Following Placement | . \ 
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Case Number Intervention. Intercepts | ° Regression Line Slopes | - 
Prior t6 Following Prior to Following 
Placement Placement Placement  Eiecenent 


*Increase in favor of placement _ r 


Of the six children examined through the evaluation, rive thilqren 
(Cases 1,+«27 3; 5, and 6) exhibited learning rates after sueenk : 
chat were higher than learning rates prior to placement. One of these 
five cases showed dramatic improvement following placement (Case 1). 
These learning rate nuPeaced indicate that, the children were progressing 
at a faster rate Foviowtnd placement than: before placement and validates 
the placement of the children in the ‘exceptional child settipg... s 

Two of the six children (Cases 2 and 5) exhibited a decrease in the 
inbateentlon ve the intervention line following placement in the « % 
exceptional child program. For both of these learners placement, o€curred 
immediately Pelowing the Christmas vacation of two weeks. The time’ 
without instruction may have caused, the depressed achievement scores. - 
Another possible explanation niGhe be that the learners had difficulty 
alsueeing to the changes in environment caused by the, placement, Case 
2, however; did not exhibit low sdaptive benavion, a 
Two cases (Cases 3 and 5) had high rates of absenteeism,*; The 


” i ; 
absenteeism probably contributed to the fluctuation in the data points 


as well as to a depression of the total achievement scores. 
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Advantages fof the Methodology 
i e ae’ 4 


Several advantages of the nethodology “ere noe, at. the conclusion . 
of the study. First, the time- series design was’ aarbicdiariy appropriate 
for research or evaluation in’ special education where\ random ‘assignment ° 


of children is seldom possible, and the possibility of regression 


e 


effects very strong. The time-series aSeigh: controls all threats to 


internal validity except history. Utilizing. the design repeatedly on 


an individual basis with the ‘treatment introduced at different. Vines. 


a, 


. Lnepedsed the weeappiiees of concluding that the treatment and’ not 
other factor's accounted for changes in the performance of ‘the children. | 
- The second advantage noted was the Tone RECO EE aia provided in 


the study. Typical Bese ee designs provide single’ measures prior to 


and followjng treatment. Collecting data ona monthly basit provided 
an indication of academic performance throughout-the vear. Individyal 
learning profiles could, be examined. me determine the effects of 


significant, events such eee eee absenteeism, or, known emotional 


distress, as well as the treatment effects. 7 . y, 


vs 


se : . a le . ‘ 4 
An additional benefit of the methédology was the use of criterion- \ 


‘ 
referenced measures. A common,criticism of research gtudies is that 
3 : l. en 
- the measures used did not adequately ,reflect the teaching program in ~ Be 
. } 
i 


f r ‘ . 
their school system. The development of a criterion-referenced test ° 
* ° 


based on local objectives, ensured the validity of the measure’ for thee!) / 
population being tested. Lt ”, SS red 


_ The fourth advantage of the methodology was the utility, of, the on we 


data for ‘purposes other than research or svaluation: The indiv¥qual 
profiles provided supplemental information on academe. weepornianae 

° ‘ ° : a’ tye r 2 
which could be helpful in making placement decisions for the following 
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year. a - 9 . os i: 
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The major ee eae of ‘the methodology as the time ‘involved 
in\the administration of the measures. Administration of the test 


required an average of 30 minutes. ‘For the 37 high-risk children 


tested, an average of 18.5 Heo hOUsS be month was devoted to the 


. 
’ 


administration along. When preparation and lqyistics were included 
Apes Ld ‘ ‘ 
in the time .estimates, a more realistic figure of 40 man-hours per , 


month was derived for the completion of the .testing, schedule. 
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; ny * © ‘ ( 
The assumptions necessary for interpretation of time-series data 


was a second problem detected, in utilizing this methodology. Ss pagOe 


assumption was that the criterion- ~referenced test Scores were equal- 


~ 


‘interval. Thus, it was “necessary to assume that resongnition of the 


letter K represented the same amount of learning as following a set of 
: ' - 


' : 
directions consisting of four independent activities. Use of regression 


', | slopes to.analyze the data introduced a second assu™ption, that the 


learning rate of the children would be a linear function of time: 


ink Saal this assumption is readily made by researchers, it may be’ 
“ay 


.o particularly aaa in special education where. spurts and, lulls in - 
) ) Maghing | rates are ‘frequently éncountered. rae 

“ si J 
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oe ‘S. oa third disadvantage of the methodology was the difficulty in - , 


Fitna vane data over children. Since treatment was introduced at 


ad 


*. are times for ail sik of: the children, summarization presented 


{ Vbopiens. Although the data could have been summarized by 


6a. ning, treatments for each oe the six eReSht the resulting realign: | 


ment of chaedriabiane prior ‘to ‘end eolicwiag placement produces missing 


ata cells that make, analybig questionable. If a sufficient number of 


-.= 


chjldren were placed between ay set of two observations, Sommarization 


_ Tnight be ee ‘ 5 


Although the number of cases in the DMSE evaluation was too small ~ 


to provide conclusive evidence, ine ee appeared to be 


~~ 
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sufficiently feasible and sensitive to detect academic growth resulting. 


from.placement in.exceptional child programs. Further studiés of this 


pane Ae be conducted to justify the additional resources |needed 


to maintain exceptional child education, 
as : , 7 
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